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Examining  ATC  Operational  Errors  Using 
THE  Human  Factors  Analysis  and  Classieication  System 


INTRODUCTION 

The  Federal  Aviation  Administration  (FAA)  is  charged 
with  maintaining  the  National  Airspace  System  (NAS) — a 
vast  and  increasingly  complex  transportation  system. 
Highly  skilled  air  traffic  control  specialists  (ATCSs) 
interact  with  a  matrix  of  radars,  computers,  and  com¬ 
munication  systems  to  ensure  the  safe  and  efficient 
operation  of  aircraft.  ATCSs  follow  established  rules 
and  procedures  to  separate  aircraft;  when  separation  is 
not  maintained  an  operational  error  (OE)  is  recorded. 
OEs  are  defined  as: 

An  occurrence  attributable  to  an  element  of  the  air  traf¬ 
fic  system  in  which:  (1)  less  than  the  applicable  separation 
minima  results  between  two  or  more  aircraft,  or  between 
an  aircraft  and  terrain  or  obstacles  (e.  g. ,  operations  below 
minimum  vectoring  altitude  (MVA);  equipment  /  person¬ 
nel  on  runways),  as  required  by  FAA  Order  7110.  65  or 
other  national  directive;  or  (2)  an  aircraft  lands  or  departs 
on  a  runway  closed  to  aircraft  operations  after  receiving 
air  traffic  authorization.  (3)  an  aircraft  lands  or  departs  on 
a  runway  closed  to  aircraft  operations,  at  an  uncontrolled 
airport  and  it  was  determined  that  a  NOTAM  regarding 
the  runway  closure  was  not  issued  to  the  pilot  as  required 
(pg.  5-1,  FAA  Order  7210.  56). 

This  report  summarizes  the  findings  of  a  study  to  ex¬ 
amine  the  underlying  human  causes  associated  with  ATC 
OEs.  The  study  involved  three  phases.  First,  a  literature 
search  was  conducted  to  identify  aviation-related  human 
error  models  and  taxonomies.  Second,  candidate  error 
models  and  taxonomies  were  evaluated  to  determine  their 
relative  strengths  and  weaknesses  regarding  their  applica¬ 
tion  to  ATC  OEs.  Third,  the  selected  candidate  was  used 
to  examine  the  items  reported  as  causal  factors  on  archival 
OE  reports.  We  hoped  that  this  method  would  help  to 
clarify  the  underlying  human  errors  in  OEs. 

This  study  supports  the  FAA’s  National  Aviation  Re¬ 
search  Plan  for  developing  enhanced  measures  of  human 
performance  and  increasing  the  understanding  of  factors 
that  lead  to  performance  decrements  (FAA,  1999a). 
It  also  supports  the  FAA’s  Strategic  Plan  to  eliminate 
accidents  and  incidents  caused  by  human  error  (FAA, 
1999b).  In  addition,  this  study  is  responsive  to  the  Safety 
Strategic  Objective  identified  in  the  DOT  Strategic  Plan 
for  2003-2008  (DOT,  2003)  and  the  FAAs  Flight  Plan 
for  2004-2008  (FAA,  2003)  to  reduce  operational  errors 
and  runway  incursions. 


BACKGROUND 

The  FAA’s  air  traffic  control  (ATC)  safety  program  relies 
on  the  timely  and  accurate  recording  and  transmission  of 
data  about  incidents  in  the  U.  S.  airspace  that  “adversely 
affect  the  capabilities  of  ATC  facilities  to  provide  safe, 
orderly,  and  expeditious  movement  of  air  traffic”  (FAA 
Form  7210-3,  2002,  p.  4-1).  To  achieve  these  results, 
the  FAA  established  a  means  for  documenting  these 
data.  For  instance,  when  established  separation  standards 
are  not  maintained,  OEs  are  reported  to  the  FAA’s  Air 
Traffic  Service  Office  of  Evaluations  and  Investigations 
(AAT-20) .  The  existing  process  for  investigating  human 
performance  factors  associated  with  OEs  involves  a  vo¬ 
luminous  amount  of  descriptive  data  but  provides  little 
information  about  the  underlying  causes  and  remedies 
(Pounds  &  Scarborough,  2000). 

We  conducted  a  literature  review  to  identify  human 
error  models  and  taxonomies  that  captured  important 
elements  of  the  cognitive  processes  of  the  human  operator, 
the  actions  of  the  operator,  and  the  environment  in  which 
the  operator  performs .  These  three  domains  extended  from 
the  earlier  work  of  Lewin’s  (1951)  field  theory  in  which  a 
person’s  behavior  (i.  e. ,  task  performance)  was  postulated 
to  be  a  function  of  the  qualities  of  both  the  person  and 
environment  in  which  the  behavior  occurred. 

In  conducting  the  literature  review,  it  became  clear  that 
the  use  of  the  term  “model”  and  “taxonomy”  sometimes 
were  used  interchangeably.  However,  for  the  purpose 
of  this  review,  a  taxonomy  is  defined  as  a  classification 
system  that  organizes  data  into  meaningful  categories 
(Kirwan,  1992).  In  contrast  to  a  taxonomy,  a  model  not 
only  describes  the  categories  within  the  system,  but  it  also 
indicates  the  manner  by  which  the  various  components  are 
affected  by  each  other  (Shorrock,  Kirwan,  Isaac,  Anderson, 
&  Bove,  1999).  While  a  taxonomy  has  descriptive  power, 
a  model  has  both  descriptive  and  predictive  power.  This 
distinction  will  be  used  later  when  evaluating  the  strength 
and  limitations  of  a  given  model  or  taxonomy. 

Review  of  Human  Error  Models  and  Taxonomies 

Our  literature  review  was  focused  on  identifying  avia¬ 
tion-related  human  error  models  and  taxonomies  that 
could  be  applied  to  the  ATC  environment.  A  search  of 
six  aviation  technical  report  databases  was  conducted  to 
identify  the  relevant  literature.  The  six  databases  included: 
(1)  the  European  Organization  for  the  Safety  of  Air 
Navigation  Report  Database,  (2)  the  Aviation  Research 
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Laboratory  Institute  of  Aviation  Report  Database,  (3) 
the  W.  J.  Hughes  Technical  Center  Reports  Database,  (4) 
the  Civil  Aerospace  Medical  Institute  (CAMI)  Technical 
Report  Database,  (5)  the  National  Aeronautics  and  Space 
Administration  (NASA)  Technical  Report  Database,  and 
(6)  the  National  Transportation  Safety  Board  (NTSB) 
Report  Database.  From  these  sources,  articles  and  book 
chapters  were  reviewed  to  determine  if  a  given  human 
error  model  or  taxonomy  was  described  in  sufficient 
detail  so  that  an  independent  analysis  could  be  made  on 
the  relative  strengths  and  limitations  associated  with  an 
operator’s:  (1)  cognitive  process,  (2)  task-related  behaviors, 
and  (3)  environmental  conditions  (i.e.,  organizational 
context). 

Ten  candidate  taxonomies/models  were  selected, 
based  on  the  completeness  with  which  they  addressed 
each  of  the  three  categories  listed  above  (See  Table  1). 
The  authors  rated  the  ten  candidates  on  eight  dimen¬ 
sions  adapted  from  Kirwan  (1992)  that  were  designed 
for  evaluating  human  error  taxonomies  (See  Table  2). 
The  eight  dimensions  included:  comprehensiveness, 
accuracy,  consistency,  theoretical  validity,  auditability, 
resource  usage,  utility,  and  acceptability.  Each  of  the 
dimensions  and  the  corresponding  rating  scale  that  we 
developed  are  described  in  the  following  section.  It  should 
be  mentioned  that  although  the  work  of  Kirwan  (1992) 
was  used  as  a  guide,  considerable  adjustments  were  made 
to  the  definitions  and  rating  scales  to  reflect  the  specific 
needs  of  this  project. 

Evaluating  Human  Error  Models  and  Taxonomies 

Comprehensiveness  refers  to  the  extent  to  which  a  content 
domain  is  sampled.  In  this  project,  the  three  domains  of 
interest  reflected  an  operator’s:  (1)  cognitive  process,  (2) 
task-related  behaviors,  and  (3)  environmental  conditions, 
such  as  organizational  context  and  external  factors  (e.g., 
government  regulations).  A  three-point  rating  scale  was 
derived  for  judging  the  degree  to  which  each  of  the  three 
domains  was  adequately  sampled  by  a  model  or  taxonomy. 
Since  the  authors  were  not  experts  in  the  held  of  human 
error  theory,  they  relied  on  the  rationale  presented  in  the 
source  documents  for  justifying  the  domain  sampling. 
A  value  of  “1”  meant  that  a  rationale  was  presented  for 
only  one  dimension  (e.g.,  person),  a  value  of  “2”  meant 
that  a  rationale  was  presented  for  two  dimensions  (e.g., 
person  and  task),  and  a  value  of  “3”  meant  that  a  rationale 
was  presented  for  all  three  dimensions  (i.e.,  person,  task, 
and  environment). 

Accuracy  assesses  the  degree  to  which  critical  operator 
errors  are  captured  by  the  error  model  or  taxonomy.  Since 
error  reduction  was  the  goal  of  this  program  of  research, 
it  is  important  that  the  criteria  by  which  error  reduc¬ 
tion  are  judged  are  clearly  identified.  Three  factors  were 


selected  for  rating  accuracy:  the  identification  of  system 
errors  (such  as  those  detected  at  the  level  of  the  National 
Airspace  System),  the  identification  of  operator  errors 
(such  as  the  errors  committed  by  a  given  ATCS),  and  an 
empirical  validation  that  the  operator  errors  affected  the 
system  errors  (such  as  how  the  number  of  ATC  OEs  affect 
the  system  outcomes  of  safety  efficiency  and/ or  effective¬ 
ness)  .  A  three-point  rating  scale  was  employed  to  indicate 
the  presence  of  one,  two,  or  all  three  factors. 

Consistency  refers  to  the  degree  to  which  different  users 
of  the  model  or  taxonomy  arrive  at  the  same  results  when 
evaluating  the  same  material  (i.e.,  accident  reports).  In 
statistical  terms  consistency  as  used  here  is  associated  with 
inter-rater  agreement.  However,  before  inter-rater  agree¬ 
ment  can  be  assessed,  it  is  important  that  the  categories  of 
the  model  or  taxonomy  are  unambiguous  and  mutually 
exclusive.  Once  again,  a  three-point  hierarchical  rating 
scheme  was  employed  to  assess  consistency:  A  “  1  ”  was  as¬ 
signed  when  mutually  exclusive  categories  were  defined,  a 
“2”  was  assigned  when  an  empirical  measure  of  inter-rater 
agreement  was  provided  for  a  single  study,  and  a  “3”  was 
assigned  when  empirical  measures  of  in  ter- rater  agreement 
were  provided  across  multiple  studies  using  different  raters. 
It  should  be  noted  that  since  this  dimension  is  associated 
with  the  calculation  of  a  statistic,  all  things  being  equal, 
the  error  model  or  taxonomy  with  the  highest  measure 
of  inter-rater  agreement  would  be  chosen. 

Theoretical  validity,  as  used  here,  addresses  the  three 
phases  of  theory  development:  description,  prediction, 
and  explanation  (Kaplan,  1 964) .  In  the  description  phase, 
the  phenomenon  of  interest  is  defined  in  a  way  that  allows 
measurement.  Given  that  error  taxonomies  are  developed 
for  the  purpose  of  identifying  error  types  and  tracking 
changes  across  time,  taxonomies  tend  to  reside  within  the 
descriptive  phase  of  theory  development.  The  prediction 
phase  is  associated  with  identifying  important  variables 
that  predict  the  criterion  of  interest.  This  requires  that 
both  the  predictor  and  criterion  are  operationally  relevant. 
Given  that  models  propose  a  casual  sequence,  they  are 
well  suited  for  a  test  of  their  predictive  validity. 

The  last  phase  of  theory  development  deals  with 
the  power  of  a  theory  to  explain  why  the  phenomenon 
occurs  (i.e.,  an  empirical  validation  of  the  causal  chain 
of  events).  However,  instead  of  using  just  words,  when 
an  error  occurs  the  theory  can  be  used  to  explain  both 
the  necessary  and  sufficient  conditions  surrounding  the 
event.  By  necessary,  we  mean  that  when  the  elements  of 
the  predictors  (i.e.,  hypothesized  causes)  are  absent,  the 
criterion  of  interest  -errors-  is  also  absent.  By  sufficient, 
we  mean  that  when  the  elements  of  the  predictors  are 
present,  the  criterion  of  interest  is  also  present  (Swartz, 
1997).  Since  it  is  difficult  to  manipulate  variables  in 
an  operational  environment,  theoretical  validity  at  the 
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Table  1.  Error  Models  and  Taxonomies. 


Name 


Source 


Description 


Error  Taxonomy 
Human  Factors 
Analysis  and 
Classification  System 
(HFACS) 


Wiegmann  &  Shappell  (2003)  Identifies  and  organizes  latent  errors 

using  a  hierarchical  structure  involving 
organizational  influences,  unsafe 
supervisory  actions,  preconditions  for 
unsafe  acts,  and  unsafe  acts. 


Violation  Taxonomy  Mason  (1997) 


A  Human  Error  Bagnara  et  al.  (1991) 

Taxonomy  based  on 

Cognitive  Engineering 

&  Social  &  Occupational 

Psychology 


Identifies  the  main  organizational 
factors  which  might  promote  violations, 
and  management  strategies  that  could 
help  to  eliminate  or  reduce  these  factors 
by  addressing  the  motives  behind  them. 

Identifies  four  categories  of  conditions 
affecting  the  state  of  a  human  system: 
human  performances,  decision  making, 
socio-organizational  conditions,  and 
external  situations. 


Latent  Error  Model  of  Reason  (1990)  Identifies  four  fundamental  elements  of 

Accident  Causation  all  organizations  that  must  work 

together  harmoniously  if  efficient  and 
safe  operations  are  to  occur:  corporate 
decision  makers,  line  management, 
psychological  precursors  of  unsafe  acts, 
and  unsafe  acts.  Within  each  of  the 
elements  are  latent  (hidden)  and  active 
failures  that  represent  vulnerability 
points  for  the  creation  of  unsafe  acts. 


Taxonomy  for 
Describing  Human 
Malfunctions 

Rasmussen  (1986) 

Identifies  factors  that  shape 
performance,  the  contextual  factors 
associated  with  error,  characteristics  of 
the  task  being  performed,  and  the 
classification  of  error  modes. 

Pyramid  Model 

Isaac  (1995) 

Classifies  human  error  within  an  air 
traffic  management  environment. 

composed  of  three  levels:  the  top  level 
(representing  individual  factors),  the 
middle  level  (representing  task 
characteristics),  and  the  bottom  level 
(representing  the  organizational 
influences). 


(Continued) 
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Table  1.  Continued 


Name 

Souree 

Deseription 

Error  Model 

Aviation  Safety  and 

Human  Reliability 

Analysis  Method 
(ASHRAM) 

Miller  and  Forester  (2000) 

Allows  aviation  researehers  to 
analyze  aviation  mishaps  that 
involve  human  errors  in  ways 
that  aeeount  for  the  operational 
eontext,  erew  expeetations, 
training,  airframe-related 
human-system  interfaees,  and 
erew  resouree  management. 

A  Teehnique  for  Human  Error 
Analysis  (ATHEANA) 

Cooper  et  al.  (1996) 

Performs  a  human  reliability 
analysis  in  the  eontext  of 
probabilistie  risk  assessment. 
ATHEANA  is  based  on  an 
understanding  of  why  human- 
system  interaetion  failures 
oeeur  as  opposed  to  behavioral 
and  phenomenologieal 
deseription  of  operator 
responses. 

A  Teehnique  for  Human  Error 
Assessment  (THEA) 

Poeoek,  Wright,  &  Harrison 
(1999) 

Used  by  interaetive  system 
designers  and  engineers  to  help 
antieipate  human-maehine 
interaetion  failures.  The 
teehnique  employs  a  eognitive 
error  analysis  based  on  an 
underlying  model  of  human 
information  proeessing. 

A  Dynamie  Reliability  Teehnique 
for  Error  Assessment  in  Man- 
Maehine  System  (DREAMS) 

Caeeiabue  (1993) 

Identifies  the  origin  of  human 
errors  in  the  dynamie 
interaetion  of  the  operator  and 
the  plant  eontrol  system. 

Human  error  probabilities  are 
eombined  with  the  prob¬ 
abilities  of  system  failures  in 
order  to  obtain  an  overall 
probabilistie  safety  assessment 
(PSA)  for  the  whole  plant. 
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Table  2.  Ratings  of  Error  Taxonomies  and  Models 


CIh 

O 

o 

c 


Pi 


> 


o 

U 


o 

o 

c 


o 

U 


u 

O 

C/!) 


c3 


CN 


CN 


<D 

Oh 

05 

§ 


(D  O 


GO 

u 


ca  <: 
H  |Jh 

ffi 

o 


(N  CN 


CN  <N 


(N  <N 


(N  <N 


CN  (N  <N 


CO  (N  CN 


r- 

c\ 

o\ 


o 


a 

o 

SP 

o 

X 

cd 

H 

SP 

o 


> 


ON 

On 


a 

c3 

CP 

PJ) 

c3 

PP 


SP 

o 

-TD 
O  22 

w 


>> 

a 

o 
s 
o 
X 
,  03 

C  H 


§ 


3 

ffi 


a 

bD  S 

.a  -2 

^  PP 
^  S 

(1>  Qh 

.2  g 

W)  O 

^  O 

“S 

> 


•  Ch  ^ 


tJ) 

o 


2  M 


s  o 

r? 

o  ^  ^ 


O 

>> 


CN 


CN 


CN 


(N 


4h 

O 

o 


SP 

o 


S  O 


CN 


CN 


CN 


CO  (N  CN 


CN  CN  CN  CN  CN 


CN 


CN  CO  CO  04  04 


04  04  04  04 


04 


04 


04 


NO 

OO 

ON 

o 

ON 

s 

ON 

sp 

lO 

o 

On 

c/: 

ON 

, 

c/: 

SP 

o 

ps 

zrt 

e 

o 

cP 

c/: 

cd 

o 

cd 

cd 

Pi 

Pi 

c/: 

1— 1 

§ 

a 

=s 

t/: 

C 

>,  PD  O 


c2  ffi 


<D 

O 


04 


O 

o 

o 

04 


(D 

S-H 

O 

Ph 

"O 


04 


ON 

ON 


o 

S-T 

o 

CIh 

o 

o 

u 


04 


04 


o 

t/J 

'G 

c3 

ffi 


4P 

PO 


y  ON 
g  ON 

o  a: 

CLh  O 


W  3 

3-^0 

2  -a  c 
a  .3 

"2 

‘g 

13 

< 

w 

< 

“  'o 
cd 

o  u  ^ 

X  "3 

^  *3 

H  Q  2 

cd 

>> 

5  K 

ffi 

H 

w 

ffi 

hj  <: 

CP 

o 

<c 

H 

a 

w 

04 


04 


CO 

ON 

ON 


=s 

-O 

.a 

‘3 

o 

CP 

u 


05 


< 


5 


Com  =  Comprehensiveness  ,  Aee.  =  Aeeuracy,  Con.  =  Consisteney,  Th.V.  =  Theoretical  Validity,  And.  =  Auditability 
Res.  =  Resource  Usage,  Utility  =  Utility,  Accept.  =  Acceptability. 


explanation  phase  is  seldom  achieved  in  total.  Using  a 
three-point  hierarchical  scale,  a  “1”  was  assigned  when 
the  definitions  were  empirically  measured,  a  “2”  was  as¬ 
signed  when  predictive  power  was  demonstrated,  and  a 
“3”  was  assigned  when  a  causal  sequence  was  tested  for 
necessary  and  sufficient  conditions. 

Auditability  refers  to  the  ease  by  which  another  person 
can  illustrate  and  discuss  the  rationale  behind  the  results 
of  the  classifications  or  model  predictions.  It  is  important 
to  note  that  the  ratings  associated  with  auditability  were 
dependent  on  the  information  reported  in  the  available 
literature  and  do  not  necessarily  refiect  the  ratings  that 
would  be  achieved  if  the  authors  had  access  to  the  “user” 
manuals.  Once  again,  a3-point  hierarchical  scale  was  used 
to  assess  this  dimension.  A  rating  of  “1”  was  assigned  to 
sources  that  provided  just  a  conceptual  view  of  an  error 
model  or  taxonomy.  A  rating  of  “2”  was  assigned  to 
sources  that  provided  steps  or  checklists  that  should  be 
followed  when  using  a  specific  error  model  or  taxonomy. 
A  rating  of  “3”  was  assigned  to  sources  that  not  only 
provided  checklists  but  also  incorporated  examples  from 
case  histories  to  illustrate  the  kinds  of  judgments  required 
of  the  user  of  a  given  model  or  taxonomy. 

Resource  Usage  is  defined  by  the  amount  of  time, 
money,  and  human  resources  necessary  to  develop  and 
maintain  the  infrastructure  to  support  the  error  model  or 
taxonomy.  Since  none  of  the  material  reviewed  provided 
quantifiable  information  on  resource  usage,  a  three-point 
subjective  scale  was  developed  based  on  how  much  effort 
would  be  required  to  convert  the  FAA  OE  database  so 
that  it  would  be  compatible  with  a  given  error  model  or 
taxonomy.  The  three  levels  included:  large  amount  of 
adaptation  required  (rating  of  “1”),  moderate  amount 
(rating  of  “2”),  and  minimal  (rating  of  “3”). 

Utility  is  based  on  the  needs  of  the  user.  In  this  case, 
the  needs  of  the  user  are  two-fold,  to  identify  and  reduce 
the  number  of  OEs.  Utility  differs  from  auditability  in 
that  the  latter  is  concerned  with  being  able  to  document 
what  was  done,  and  the  former  is  concerned  with  whether 
what  was  done  helped  to  reduce  errors.  A  three-point 
scale  was  developed  based  on  the  ability  of  an  error  tax¬ 
onomy  or  model  to  specify  error  reduction  measures.  A 
rating  of  “1”  was  given  when  errors  were  identified,  but 
no  guidance  was  provided  for  reducing  them.  A  rating 
of  “2”  was  used  when  general  error-reduction  guidelines 
were  presented.  Finally,  a  rating  of  “3”  was  used  when 
guidelines  were  provided  for  reducing  specific  classes  or 
types  of  errors. 

Acceptability  depends  on  how  referent  groups  have 
judged  the  value  of  a  given  model  or  taxonomy.  The  two 
referent  groups  of  interest  in  this  study  are  academicians 
and  safety  practitioners.  Aspects  of  both  referent  groups 
were  integrated  into  a  three-point  hierarchical  scale.  A 


rating  of  “1”  was  given  if  the  error  model  or  taxonomy 
received  afavorable  review  in  the  scientific  literature  about 
human  error.  A  rating  of  “2”  was  granted  if  in  addition  to 
receiving  a  favorable  review,  the  error  model  or  taxonomy 
was  also  successfully  implemented  in  an  error  reduction 
program  within  a  command  and  control  environment 
(e.g.,  a  nuclear  power  plant  or  a  military  operations  control 
center).  If  the  command  and  control  environment  was 
aviation  related,  the  rating  was  increased  to  a  “3.” 

Using  the  above  rating  scheme,  4  error  models  and  6 
error  taxonomies  were  rated  by  the  first  two  authors,  as 
shown  in  Table  2.  The  numerical  ratings  associated  with 
the  8  dimensions  reflected  the  consensus  of  the  authors. 
Although  an  argument  could  be  made  about  any  given 
rating,  the  overall  trend  revealed  the  following; 

1.  There  was  little  if  any  variability  in  the  ratings  as¬ 
sociated  with:  accuracy,  consistency,  auditability,  and 
utility.  Thus,  each  of  the  10  articles  included: 

a.  System  and  operator  error  identification. 

b.  Definitions  of  mutually  exclusive  categories 
(HFACS  also  included  measured  inter-rater  reli¬ 
ability). 

c.  User  outline/ checklist  describing  the  steps  to  follow 
for  implementing  an  error  model  or  taxonomy. 

d.  General  error  reducing  guidelines. 

2.  Rating  variability  for  theoretical  validity  and  resource 
usage  was  explained  by  the  type  of  technique  used  (i.e., 
error  model  vs.  error  taxonomy). 

a.  All  the  error  models  had  higher  theoretical  va¬ 
lidity  ratings  when  compared  with  all  the  error 
taxonomies.  It  should  be  noted,  however,  that  the 
higher  ratings  for  the  error  models  was  due  to  their 
potential  for  testing  predictive  validity  rather  than 
empirically  demonstrating  the  predictive  power  of 
a  given  technique. 

b.  All  the  error  taxonomies  were  rated  as  requiring 
fewer  resources  to  implement  as  compared  to  all 
the  error  models.  Again,  the  source  documents  did 
not  provide  empirical  data  for  direct  comparisons. 
That  being  the  case,  the  ratings  reflected  our  best 
judgments  based  on  the  available  materials. 

3.  The  dimensions  of  completeness  and  acceptability 
were  the  only  two  dimensions  whose  rating  variability 
proved  to  be  source  specific.  That  being  the  case,  these 
two  dimensions  provided  the  most  useful  informa¬ 
tion  with  regard  to  selecting  a  given  error  model  or 
taxonomy  for  use  in  a  study  of  the  underlying  human 
factor  causes  of  OEs. 

a.  FIFACS  and  Reason’s  Latent  Error  Model  were 
the  two  highest-rated  error  taxonomies. 

b.  ASHRAM  emerged  as  the  highest-rated  error 
model. 
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Given  that  error  models  and  taxonomies  differed 
on  their  relative  strengths  and  limitations,  the  authors 
decided  to  select  one  candidate  from  each  method  for 
further  examination.  HFACS  was  selected  to  represent 
the  error  taxonomies  and  ASHRAM  was  selected  to 
represent  the  error  models.  HFACS  was  selected  not 
only  because  of  the  results  of  Table  2  but  also  because 
its  framework  incorporates  the  work  of  Reason  (1990) 
and  Rasmussen  (1986),  both  of  which  were  highly  rated 
taxonomies.  Similarly,  ASHRAM  was  selected  because 
it  adapted  the  ATHEANA  model  (also  highly  rated)  for 
use  in  the  aviation  industry.  A  synopsis  of  both  HFACS 
and  ASHRAM  is  presented  next. 

The  Human  Factor  Analysis  and  Classification 
System  (HFACS) 

The  Human  Factors  Analysis  and  Classification  System 
(HFACS;  Wiegmann  &  Shappell,  2003)  classifies  hu¬ 
man  errors  and  the  causal  factors  associated  with  them. 
Originally  developed  for  aviation  accidents  and  mishaps, 
HFACS  has  been  adapted  to  other  domains,  such  as 
mechanical  and  medical  errors  (ibid).  HFACS  has  strong 
linkages  to  Reason’s  (1990)  model  of  latent  and  active 
errors  (see  Figure  1)  as  they  relate  to  organizational  influ¬ 
ences,  unsafe  supervision,  pre-conditions  for  unsafe  acts 
of  operators  (i.e.,  factors  that  affect  an  individual’s  mental 
and  physical  behavior),  and  the  unsafe  acts  themselves. 
Active  errors  are  related  to  the  performance  of  operators 
in  complex  systems  that  have  an  immediate  impact  on 
the  system,  in  contrast  to  latent  (or  hidden)  errors  of 
designers  and  managers.  Whereas  the  active  errors  of  the 
operator  attract  the  most  attention  during  an  accident 
investigation.  Reason  (1990)  noted  that  the  latent  errors 
of  the  organization,  supervisor,  and  the  preconditions  of 
unsafe  acts,  may  pose  the  greatest  risk  to  system  safety. 
This  is  because  latent  errors,  unless  they  are  discovered, 
will  continue  to  remain  in  the  system  even  after  rectifying 
errors  committed  by  an  individual  operator. 

Figure  1  illustrates  the  concept  of  latent  errors.  Each 
of  the  four  layers  has  “holes”  (i.e.,  latent  errors),  which 
represent  potential  deficits  or  vulnerabilities  in  system 
safety.  Under  normal  conditions,  the  holes  are  out  of 
alignment,  which  means  a  deficit  in  one  layer  of  the 
system  is  compensated  by  some  protection  in  another 
layer.  For  example,  when  a  pilot  catches  the  mistake  of 
a  co-pilot  or  an  aircraft  mechanic.  However,  when  the 
holes  are  in  alignment,  it  means  that  a  deficit  in  one 
layer  of  the  system  is  exacerbated  by  a  deficit  in  another 
layer.  An  example  of  multiple  latent  errors  in  alignment 
is  a  situation  where  a  supervisor  discovers  that  a  pilot  is 
not  following  the  Federal  Aviation  Regulations  (FARs) 
or  standard  operating  procedures  (SOPs)  and  elects  not 
to  do  anything  about  it  (i.e..  Unsafe  Supervision).  The 


pilot  continues  to  follow  bad  practices  (i.  e.  ,  Precondi¬ 
tion)  until  one  day  he  commits  several  “Unsafe  Acts”  by 
flying  under  visual  flight  rules  (VFR)  into  instrument 
meteorological  conditions  (IMC),  that  results  in  a  con- 
trolled-flight-into-terrain  (CFIT)  accident. 

The  causal  sequence  depicted  by  the  arrow  in  Figure  1 
suggests  that  an  unsafe  act  is  the  final  failure  in  a  series  of 
deficits  that  begins  at  the  organizational  level.  However, 
when  conducting  accident  investigations,  the  starting  point 
begins  with  the  unsafe  act  committed  by  the  operator.  By 
asking  “why”  an  unsafe  act  was  committed,  investigators 
can  follow  the  accident  chain  of  events  through  each  of 
the  four  levels  and  uncover  the  direct  and  indirect  influ¬ 
ences  on  an  operator’s  actions.  As  one  might  imagine,  the 
discovery  of  latent  errors  requires  diligent  search. 

Figure  2  shows  how  HFACS  extended  Reason’s  model 
by  systematically  identifying  the  vulnerability  points 
(i.e.,  latent  errors)  at  each  level  in  the  system,  and  then 
demonstrating  how  this  analysis  provided  additional  in¬ 
formation  about  the  human  factor  causes  associated  with 
aviation  accidents.  (See  Wiegmann  &  Shappell,  2003,  for 
a  thorough  description  of  the  causal  categories  associated 
with  each  error  level.)  For  instance,  organizational  influ¬ 
ences  are  vulnerable  to  errors  associated  with  resource 
management,  organizational  climate,  and  organizational 
processes.  Resource  management  includes  the  manage¬ 
ment  ofhuman  resources,  financial  allocations,  equipment, 
and  facilities.  Organizational  climate  is  broken  down  by 
its  management  structure,  policies  and  procedures,  and 
the  underlying  culture.  Organizational  processes  are  the 
operations,  procedures,  and  safety  oversight  that  exist 
within  the  organization. 

A  more  complete  illustration  of  HFACS  is  shown  in 
Figure  3. 

Error  vulnerabilities  at  the  supervisory  level  manifest 
themselves  in  the  form  of  unsafe  supervision.  For  example, 
when  supervisors  fail  to  provide  guidance  or  accurate 
information  to  the  workforce  in  a  timely  fashion,  correct 
known  problems,  or  willfully  allow  personnel  to  violate 
safety-related  rules  and  regulations,  this  creates  a  potentially 
hazardous  environment  for  the  operation. 

Preconditions  of  unsafe  acts  include  environmental 
factors,  personal  factors,  and  the  physical,  mental,  and 
physiological  condition  of  the  operator.  Examples  of 
the  latter  include  physical  fatigue,  mental  fatigue,  and 
effects  of  over-the-counter  medications.  Whereas  envi¬ 
ronmental  factors  represent  those  forces  outside  of  the 
individual  that  affect  performance  (such  as  the  weather 
and  cockpit  displays),  personal  factors  refer  to  events  that 
are  brought  on  by  the  operator.  These  events  include  the 
use  or  misuse  of  crew  resources  and  factors  that  relate  to 
personal  readiness,  such  as  staying  up  late  the  night  before 
an  early  departure. 
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Figure  2.  The  Human  Factors  Analysis  and  Classification  System  (HFACS) 
Source:  Adapted  from  Wiegmann  &  Shappell  (2003) 


©Shappell  &  Wiegmaiin,  2002 


Figure  3.  The  Human  Factors  Analysis  and  Classification  System  (HFACS) 
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Unsafe  acts  are  operator  actions  involved  in  producing 
an  error.  The  erroneous  actions  are  classified  as  either 
unintentional  errors  or  willful  violations.  The  subdivisions 
of  unintentional  errors  are  an  adaptation  of  Rasmussen’s 
Skills,  Rules  and  Knowledge  (SRK)  error  model  (Rasmus¬ 
sen,  1986).  The  subdivisions  include  decision  errors, 
skilled-based  errors,  and  perceptual  errors.  In  contrast  to 
errors,  violations  are  willful  deviations  from  known  rules 
and  procedures.  A  more  comprehensive  overview  of  the 
HFACS  categories  appears  in  Appendix  A. 

Aviation  Safety  and  Human  Reliability  Analysis 
Method  (ASHRAM) 

The  Aviation  Safety  and  Human  Reliability  Analysis 
Method  (ASHRAM)  developed  by  Miller  and  Forester 
(2000)  belongs  to  a  class  of  probability  risk  assessment 
models  (PRA)  that  incorporates  the  reliability  of  human 
performance,  commonly  referred  to  as  a  Human  Reli¬ 
ability  Assessment  (HRA).  ASHRAM  is  an  extension  of 
A  Technique  for  Human  Error  Analysis  (ATHEANA) 
originally  developed  for  the  U.  S.  Nuclear  Regulatory 
Commission  (NRC).  In  their  report  on  the  development 
of  ASHRAM,  the  authors  noted  that,  although  there 
were  similarities  between  the  nuclear  power  and  aviation 
industries,  there  were  enough  differences  to  warrant  the 
development  of  a  HRA  method  unique  to  aviation. 

The  reported  similarities  between  the  nuclear  power 
and  aviation  industry  included: 

1 .  Highly  technological  systems 

2.  High  consequences  of  failure 

3.  Very  few  significant  failure  events 

4.  Governmental  regulation  of  hardware  and  operations 

5.  Small,  highly  qualified  crew  in  control 

6.  Simulators  used  in  crew  training 

7.  Dependence  upon  displays  for  information  about  the 
environment. 

Differences  between  the  two  industries  are  shown  in 
Table  3.  Appendix  B  provides  additional  background  on 
PRAs,  HRAs,  and  the  development  of  ATHEANA. 

As  shown  in  Figure  4,  ASHRAM  is  a  causal  model 
of  a  pilot’s  cognitive  processes,  which  consists  of  the 
error  forcing  context,  error  mechanisms,  and  unsafe  or 
contributory  actions.  The  Error  Forcing  Context  (EFC) 
is  divided  into  two  categories,  those  that  influence  the 
pilot  and  those  that  influence  the  aircraft.  Both  types  of 
influences  serve  as  input  to  the  pilot’s  environmental  per¬ 
ceptions.  Pilot  EFCs  consist  of  six  channels  of  input:  (1) 
information  from  maps  and  manuals,  (2)  environmental 
cues  (i.  e. ,  things  happening  outside  of  the  aircraft)  such 
as  traffic  (TF)  and  weather  (WX),  (3)  data  link  (wire¬ 
less  transfer  of  information  such  as  the  listing  of  pilot 
reports  [PIREPs]  and  electronic  communications  from 


air  traffic  control,  ATC),  (4)  displays  and  controls  (flight 
and  systems  instrumentation,  control  yoke,  trim  wheels, 
switches  and  cockpit  computer  input  devices),  (5)  radio 
communications  from  ATC,  other  planes,  and  intercom 
with  other  crew  members,  and  (6)  verbal  communications 
with  other  crew  members.  The  design  of  the  aircraft  and 
the  laws  of  physics  as  well  as  the  control  actions  of  the 
pilot  determine  aircraft  EFCs. 

As  with  ATHEANA,  the  basic  cognitive  model  detail¬ 
ing  the  mechanisms  of  pilot  error  is  based  on  the  work  of 
Woods,  Johannesen,  Cook,  &  Sarter  (1994),  Woods  & 
Paterson  (2000) ,  Roth,  Mumaw,  &  Lewis  ( 1994) ,  Mumaw 
and  Roth  ( 1 992) ,  and  Reason  (1990).  The  model  consists 
of  three  classes  of  cognitive  function  that  are  interactive 
and  non-sequential:  (1)  Environmental  Perception,  (2) 
Reasoning  and  Decision  Making,  and  (3)  Action.  These 
error  producing  mechanisms  are  further  augmented  by 
operator  conditions  such  as  stress  and  fatigue  and  the 
training  received  concerning  aircraft  design  and  operat¬ 
ing  procedures. 

Environmental  perceptions  consist  of  perceptual 
processes,  attention,  detection,  recognition,  monitoring, 
interpretation  of  environmental  cues,  and  the  overall 
understanding  of  the  state  of  the  aircraft/environmental 
system.  Although  the  elements  that  comprise  environ¬ 
mental  perceptions  are  similar  to  the  elements  comprising 
what  is  commonly  referred  to  as  “situation  awareness” 
(SA) ,  Miller  and  Forester  (2000)  note  that  there  is  enough 
controversy  over  the  definition  of  SA  that  it  became 
necessary  to  avoid  that  term.  For  more  information 
about  SA  and  the  controversy  surrounding  it,  the  reader 
is  referred  to  Endsley  (1995),  Flach  (1995),  and  Sarter 
and  Woods  (1991). 

Reasoning  and  decision  making,  as  defined  by  Miller 
and  Forester  (2000),  is  the  cognitive  or  thinking  process 
that  includes  awareness  and  deduction  of  unsafe  or  danger¬ 
ous  conditions,  remembering  situation-specific  training, 
deciding  to  follow  recommended  procedures,  planning 
flight  navigation,  diagnosis  of  trouble  symptoms,  deciding 
how  to  respond  to  situations,  problem  solving,  and  novel 
or  creative  use  of  existing  tools  or  symptoms. 

The  pilot’s  actions  are  the  control  inputs  to  the  airframe, 
operation  of  control  hardware  in  the  cockpit,  commu¬ 
nications  to  crew  and  passengers,  and  any  other  overt 
physical  behaviors.  As  shown  in  Figure  4,  these  actions 
may  produce  unsafe  or  contributory  actions. 

A  pilot’s  unsafe  actions  are  the  overt  acts  of  commis¬ 
sion  or  omission  that  are  taken  by  the  pilot  and/or  crew¬ 
members  that  lead  to  a  degradation  in  safety.  However, 
Miller  and  Forester  (2000)  emphasize  that  the  individuals 
involved  may  not  know  they  are  committing  an  unsafe 
act,  and  instead  may  be  performing  actions  that  make 
sense  to  them  at  the  time.  The  latter  then  are  referred  to 
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Table  3.  Summary  of  the  Major  Differences  Between  Nuclear  Power  and  Aviation  as  Reported  in  Miller, 
D.,  &  Forester,  J.  (2000). 


Topics 


Nuclear  Power 


Commercial  Aviation 


Licensing  agency 

Potential  accident  consequences 

Incentives  to  operate 
w/inadequate  safety 
Reports  of  errors  and  near 
misses — ^human  error 
probabilities  (HEPs)  available 

Contact  with  help  in  emergencies 


Normal  operations 


Minimum  elements  needed  for 
mission:  critical  functions 


Physical  inertia 


Speed  of  system  response 

Feedback  from  system  to  control 
inputs 


Emergency  operation  written 
guidance 

Accident  sequences 

Transient  conditions 

Activation  of  emergency 
subsystems 


NRC 

Extremely  High;  thousands  of 
lives 

Power  grid  needs,  profits  of 
utility 

The  industry  has  developed  an 
HEP  databank,  called 
NUCLARS,  but  participation  has 
been  minimal 

Shift  Technical  Advisor,  Incident 
team  at  Emergency  Operations 
Center,  Technical  Evaluation 
Center 

Continuous,  mostly  supervisory 
control,  with  periods  of  direct, 
manual  control 


Fuel,  cooling,  pressure  control, 
power  conversion  systems,  crew, 
safety  systems 

High,  with  a  few  notable 
exceptions,  such  as  large  loss-of- 
coolant  accidents  (LOCAs), 
changes  in  physics  take  place 
slowly — minutes  and  hours 
Relatively  slowly,  except  large 
LOCA  -  slow  feedback  from 
inputs 

Remote  reports  and 
instrumentation;  mostly  discrete 
readouts,  but  some  integrated 
displays,  mixture 
of  electromechanical  and 
electronic 

Written,  symptom-based 
procedures 

A  few  major  decisions  and 
actions  can  cover  several  hours 
Difficult  to  integrate  all  info  to 
construct  valid  mental  model 
Automatic,  in  most  cases,  with 
crew  notification 


Federal  Aviation  Administration 
Very  High;  hundreds  of  lives 

Meet  schedule,  passenger 
frustration,  airline  profits 
There  are  several  databases  of 
near  misses  and  accidents,  some 
are  privately  owned  by  airlines, 
others  are  public — no  known 
banks  include  HEPs 
Radio  to  radar  centers,  tower, 
airlines,  manufacturer 

Each  flight  is  a  discrete  event  and 
is  dependent  on  crew  for 
initiating  and  orchestrating. 
During  cruise  supervisory  control 
is  used. 

Flight  controls,  thrust,  cabin 
pressure  or  supplemental  oxygen, 
navigation  information,  pilot, 
communication  w/destination, 
flyable  weather 

Low,  changes  happen  rapidly — 
seconds  and  minutes 


Relatively  fast — rapid  feedback 
from  inputs 

“Seat-of-pants”,  real-time  visual, 
aural,  kinesthetic,  also 
instrumentation;  mostly  discrete 
readouts,  but  some 
integrated  displays,  mixture  of 
electromechanical  and  electronic 
“Manual  decision-making,”  and 
checklists  for  most  critical  flight 
operations 

Many  decisions  and  actions  can 
cover  only  a  few  minutes 
Easier  to  construct  mental  model 
from  discrete  displays 
Pilots  are  in  the  loop — get 
warning  displayed  and  have  to 
initiate  safety  system  response 
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Error-Forcing  Context 


Operator  Factors 
Stress,  fatigue,  skills,  knowledge. 
Tendencies  &  rules 


Figure  4.  ASHRAM  (Miller  &  Forester,  2000) 


as  contributory  actions.  For  example,  the  assumption  that 
an  aircraft  was  running  out  of  fuel  (due  to  the  erroneous 
belief  that  both  right  and  left  tanks  had  been  used)  may 
lead  to  a  forced  landing  on  inappropriate  terrain. 

Selecting  and  Testing  the  Candidate 

As  can  be  seen  from  the  previous  discussions,  both 
FiFACS  and  ASFiRAM  are  well  suited  for  the  study  of 
pilot  errors.  Thus,  the  authors  chose  between  the  two 
techniques  based  on  which  one  (1)  could  be  more  readily 
used  for  the  study  of  OEs  and  (2)  would  be  best  suited  for 
a  retrospective  analysis  of  archival  OE  data.  Both  FIFACS 
and  ASHRAM  employ  retrospective  analysis  techniques 
in  determining  the  underlying  causes  of  human  factors 
errors  among  pilots.  In  fact,  a  number  of  the  elements 
associated  with  ASHRAM’s  error  mechanisms  shown  in 
Figure  4  have  a  parallel  in  HFACS  due  to  the  common 
influence  of  Reason’s  (1990)  model  of  human  error. 

In  the  end,  the  choice  was  based  on  the  time  it  would 
take  to  develop  materials  from  each  of  the  two  techniques 
for  use  in  our  study.  Of  the  two  candidates,  we  determined 
that  HFACS  could  be  used  “as  is.”  Thus,  the  HFACS 
taxonomy  was  selected.  Later,  in  the  discussion  section, 
we  will  return  to  the  issue  of  how  HFACS  and  ASHRAM 
might  be  integrated  so  that  an  analysis  could  be  conducted 
to  assess  risk  factors  associated  with  OEs. 


METHOD 

Three  steps  were  used  to  study  the  underlying  human 
factors  causes  of  ATC  OEs. 

First,  two  coders  were  trained  to  use  of  the  HFACS 
methodology.  Second,  items  from  the  FAAs  Air  Traffic 
Control  Operational  Error/Deviation  Report  Form  7210- 
3  (FAA,  2002)  were  mapped  onto  HFACS  categories. 
Third,  once  a  reliable  mapping  was  completed,  the 
results  were  used  to  describe  the  pattern  of  human  fac¬ 
tors  causes  of  ATC  OEs  reported  for  the  five-year  period 
from  1998-2002. 

Coders 

Two  ATC  SMEs  were  selected  for  their  experience 
with  operations  in  both  terminal  and  en  route  facilities. 
Each  SME  had  more  than  15  years  of  combined  ATC 
experience  in  these  types  of  facilities.  In  addition,  both 
SMEs  possessed  specialized  knowledge  about  the  air  traf¬ 
fic  control  quality  assurance  process  and  experience  in 
identifying  the  causal  factors  associated  with  OEs.  Both 
SMEs  had  job-related  experience  using  Form  7210-3  while 
working  at  ATC  facilities.  They  also  obtained  HFACS 
experience  in  a  previous  study  requiring  that  they  use 
HFACS  subcategories  to  classify  narrative  information 
from  ATC  OEs. 
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Materials 

Operational  Error/Deviation  Report  (Form  7210-3). 
Block  68  of  Form  721 0-3  lists  potential  OE  causal  factors 
(See  Appendix  C).  The  causal  factors  are  divided  into  6 
major  categories:  (1)  Data  Posting,  (2)  Radar  Display,  (3) 
Aircraft  Observation  (Towers  only),  (4)  Communication 
Error,  (5)  Coordination,  and  (6)  Position  Relief  Briefing. 
Each  main  category  is  further  divided  into  clusters  of  items 
that  provide  greater  specification  about  the  causal  factor 
category.  Each  cluster  also  has  an  item  labeled  “other” 
but  this  item  was  not  used  in  this  study.  Overall,  a  total 
of  48  items  from  Block  68  were  used. 

The  Human  Factors  Analysis  and  Classification  System 
(HFACS) .  Coders  were  provided  copies  of  the  definitions 
for  each  HFACS  category  and  subcategories  as  shown  in 
Appendix  A.  HFACS  categories  used  for  the  classifica¬ 
tion  task  included  unsafe  operator  acts,  preconditions 
for  unsafe  acts,  unsafe  supervision,  and  organizational 
influences. 

Procedures 

The  procedures  were  divided  into  three  phases,  a  train¬ 
ing  phase  for  the  SME  coders,  a  mapping  phase,  and  a 
classification  phase  conducted  by  the  researchers. 

Training  Phase 

Each  SME  received  a  blank  copy  of  the  causal  factors 
listed  on  FAA  Form  7210-3  in  Block  68  (see  Appendix 
C)  and  the  HFACS  categories  and  definitions  (see  Ap¬ 
pendix  A).  After  reviewing  the  materials  the  SMEs  were 
asked  if  they  needed  further  clarification.  The  SMEs 
were  then  provided  with  separate  office  space  and  asked 
to  associate  each  item  in  Block  68  of  Form  7210-3  with 
an  appropriate  HFACS  category.  Once  they  completed 
their  tasks  they  were  brought  together  to  discuss  any 
problems  they  encountered.The  SMEs  were  provided  with 
an  opportunity  to  change  their  results  before  returning 
all  material  to  the  researchers. 

Mapping  Phase 

One  week  after  training,  the  SMEs  returned  to  repeat 
the  same  task  they  accomplished  during  the  training 
phase.  For  this  activity  the  SMEs  worked  independently 
and  were  not  permitted  to  discuss  their  results  until  they 
were  turned  in  to  the  researcher.  These  procedures  were 
necessary  to  calculate  a  measure  of  inter-rater  agreement 
so  that  comparisons  could  be  made  with  results  reported 
in  Wiegmann  &  Shappell  (2003).  After  the  researchers 
had  collected  the  materials,  any  differences  between  the 
two  SMEs  were  discussed  until  a  consensus  was  reached 
as  to  the  “right  answer.”  These  results  were  then  used  to 
conduct  the  classification  phase. 


Classification  Phase 

The  authors  constructed  a  Block  68  by  HFACS  fre¬ 
quency  matrix  using  a  sample  of  convenience  consisting 
of  10,754  causal  factors  extracted  from  5,011  FAA  OE 
reports  for  the  period  1998  to  2002.  The  frequencies 
associated  with  each  category  listed  in  Block  68  of  Form 
7210-3  were  then  associated  with  the  appropriate  HFACS 
category  as  identified  in  the  mapping  phase. 

RESULTS 

Matching  Phase 

Table  4  shows  the  results  of  the  matching  phase  during 
which  the  two  SME  coders  associated  each  of  the  48  items 
from  Block  68  of  Form  7210-3  with  a  corresponding 
HFACS  category.  Coefficient  Kappa  was  computed  to 
test  the  extent  to  which  the  two  SMEs  agreed  on  their 
associations.  The  resulting  kappa  of  .96  indicated  a  high 
level  of  agreement  (Fleiss,  1981).  This  value  is  somewhat 
higher  than  those  reported  by  Wiegmann  and  Shappell 
(2003).  In  those  studies  kappa  ranged  from  a  low  of  .60 
to  a  high  of  .95. 

Of  the  48  Block  68  items,  62%  (n=29)  were  associated 
with  the  HFACS  subcategory  of  skill-based  errors  and 
38%  (n=  1 8)  were  associated  with  the  HFACS  subcategory 
of  decision  errors.  It  is  interesting  to  note  that  the  SMEs 
were  unable  to  find  any  Block  68  items  that  corresponded 
to  preconditions  for  unsafe  acts,  unsafe  supervision,  or 
organizational  influences.  This  was  because  there  was  no 
place  on  Form  7210-3  to  record  this  information. 

Table  4  also  demonstrates  that  certain  item  categories 
from  Block  68  were  more  often  associated  with  a  given 
HFACS  subcategory  than  others.  For  example,  most  of 
the  items  comprising  the  data-posting  category  were  as¬ 
sociated  with  skill-based  errors.  In  contrast,  for  the  tower 
observations  and  position  debriefings  categories,  the  items 
were  predominately  associated  with  decision  errors.  Items 
in  the  radar  display  and  coordination  clusters  had  a  mixed 
association  with  skill-based  and  decision  errors. 

Classification  Phase 

Of  the  10,754  causal  factors  extracted  from  5,011  OE 
reports,  1350  (13%)  causal  factors  were  listed  as  “other.” 
Since  no  additional  information  was  provided  as  to  what 
“other”  might  be,  this  information  could  not  be  classified. 
Thus  these  data  were  dropped  from  further  analyses.  The 
remaining  9,404  causal  factors  were  categorized  according 
to  the  mapping  framework  identified  by  the  SMEs  in  the 
matching  task.  The  results  are  displayed  in  Table  5. 

Table  5  is  a  Block  68  by  HFACS  category  matrix  in 
which  row  totals  are  the  total  number  of  causal  factors 
associated  with  a  given  Block  68  category,  and  column 
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Table  4.  Form  7210-3  Causal  Factors  and  Associated  HFACS  Subcategories 


Causal  Factors 


Data  Posting  Category 

Computer  Entry  Cluster 
Ineorreet  input 
Ineorreet  update 
Premature  termination  of  data 
Input/Update  not  made 
Flight  Progress  Strip  Cluster 
Not  updated 
Interpreted  ineorreetly 
Posted  ineorreetly 
Updated  eorreetly 
Premature  removal 


Associate  HFACS 
Subcategories 


Skill-Based  Error 
Skill-Based  Error 
Deeision  Error 
Skill-Based  Error 

Skill-Based  Error 
Skill-Based  Error 
Skill-Based  Error 
Skill-Based  Error 
Skill-Based  Error 


Radar  Display  Category 

Misidentifieation  Cluster 

Failure  to  reidentify  aireraft  when  the  aeeepted  target 
identity  beeomes  questionable 
Overlapping  data  bloeks 

Aeeeptanee  of  ineomplete  or  diffieult  to  eorrelate 
position  information 

Inappropriate  Use  of  Displayed  Data  Cluster 
MODEC 
BRITE 
Confliet  alert 

Failure  to  deteet  displayed  data 

Failure  to  eomprehend  displayed  data 

Failure  to  projeet  future  status  of  displayed  data 


Skill-Based  Error 

Skill-Based  Error 
Deeision  Error 


Deeision  Error 
Deeision  Error 
Deeision  Error 
Skill-Based  Error 
Skill-Based  Error 
Skill-Based  Error 


Aircraft  Observation  (Towers  Only)  Category 

Actual  Observation  of  Aircraft 
Improper  Use  of  Visual  Data  Cluster 
Landing 
Taking  Off 
Ground  Operation 
Taxiing  across  runway 
Flolding  in  position  for  takeoff 
Phraseology 
Transposition 
Misunderstanding 
Read-back  Cluster 
Altitude 
Clearance 
Identification 
Other 

Acknowledgment 


Decision  Error 

Decision  Error 
Decision  Error 
Decision  Error 
Decision  Error 
Decision  Error 
Skill-Based  Error 
Skill-Based  Error 
Skill-Based  Error 

Skill-Based  Error 
Skill-Based  Error 
Skill-Based  Error 

Skill-Based  Error 


(Continued) 
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Table  4.  Continued 


Communication  Error  Category 

Associate  HFACS 
Subcategories 

Coordination  Category 

Area  of  incident  Cluster 

Intra-sector/position 

Skill-Based  Error 

Inter-sector/position 

Skill-Based  Error 

Inter-facility 

Skill-Based  Error 

Facility  type 

Failure  to  utilize/comply  with  precoordination  information 

Skill-Based  Error 

Improper  use  of  Information 

Exchanged  in  Coordination  Cluster 

Aircraft  identification 

Decision  Error 

Altitude/Flight  level 

Decision  Error 

Route  of  flight 

Decision  Error 

Speeds 

Decision  Error 

APREQs 

Decision  Error 

Special  instmctions 

Decision  Error 

Failure  to  Coordinate  between  Ground  and 

Local  Control  Cluster 

Crossing  active  runway 

Skill-Based  Error 

Vehicle,  equipment,  or  personnel  on  active  runway 

Skill-Based  Error 

Use  of  other  than  active  runway  for  arrival 

Skill-Based  Error 

and  departures 

Runway  closure 

Skill-Based  Error 

Position  Relief  Briefing  Category 

Employee  did  not  use  position  relief  checklist 

Decision  Error 

Employee  being  relieved  gave  incomplete  briefing 

Skill-Based  Error 

Relieving  employee  did  not  make  use  of  pertinent  data 

Decision  Error 

exchanged  at  briefing 
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totals  are  the  total  number  of  causal  factors  associated  with 
a  given  HFACS  category.  Cell  totals  are  the  frequency 
with  which  a  given  Block68  category  was  associated  with 
a  given  HFACS  category. 

By  examining  the  row  totals  we  can  see  that  of  the 
9,404  causal  factors  extracted,  5,510  (59%)  were  associ¬ 
ated  with  the  radar  display,  1,486  (16%)  were  associated 
with  communications,  961(10%)  were  associated  with 
coordination,  721(8%)  were  associated  with  aircraft  ob¬ 
servation,  583(6%)  were  associated  with  data  posting,  and 
143(1%)  were  associated  with  position  relief  briefings. 

When  the  same  Table  5  information  is  examined  by 
column  totals  we  see  that  8,093  (86%)  cases  were  clas¬ 
sified  as  skill-based  errors  and  1,311  (14%)  cases  were 
classified  as  decision  errors.  It  is  interesting  to  note  that 
these  results  are  similar  to  those  reported  for  aviation 
accidents.  For  the  period  1 990-2000, Wiegmann  and 
Shappell  (2003)  reported  that  80%  of  general  aviation 
accidents  involved  skill-based  errors  and  20%  involved 
decision  errors. 

The  cell  values  shown  in  Table  5  represent  the  interac¬ 
tion  of  row  and  column  totals  for  each  of  the  six  Block 
68  categories.  First,  of  the  583  data  posting  errors,  556 
(95%)  were  classified  as  skill-based  errors  and  27  (5%) 
were  classified  as  decision  errors.  Second,  of  the  5,510 
OEs  related  to  the  radar  display,  5,260  (95%)  were  clas¬ 
sified  as  skill-based  errors  and  250  (5%)  were  classified  as 
decision  errors.  Third,  for  OEs  related  to  problems  with 
the  Tower  controller’s  observation  of  aircraft,  all  721  of 
them  (100%)  were  classified  as  decision  errors.  Fourth, 
for  OEs  reported  as  communication  errors,  all  1,486  of 
them  (100%)  were  classified  as  skill -based  errors.  Fifth, 
of  the  961  OEs  reported  as  involving  coordination,  741 
(77%)  were  classified  as  skill-based  errors  and  220  (23%) 
were  classified  as  decision  errors.  Sixth,  of  the  143  OEs 
related  to  position  relief  briefings,  50  (35%)  were  clas¬ 
sified  as  skill-based  errors  and  93  (65%)  were  classified 
as  decision  errors. 

DISCUSSION 

The  goal  of  this  project  was  to  systematically  examine 
the  underlying  human  factor  causes  of  ATC  OEs.  This 
was  achieved  by  using  the  HFACS  framework  for  the 
ATC  environment.  One  of  the  immediate  benefits  from 
classifying  OEs  into  HFACS  categories  was  the  realiza¬ 
tion  that  the  majority  (86%)  of  OEs  were  classified  as 
skill-based  errors  and  not  decision  errors.  The  ramifica¬ 
tions  of  this  finding  affect  not  only  the  kind  of  training 
that  controllers  should  receive  following  an  OE,  but  also 
the  expectations  about  the  impact  that  ATC  decision 
support  tools  will  have  on  lowering  the  number  and 
severity  of  OEs. 


As  previously  discussed,  skill-based  errors  tend  to  be  the 
result  of  habitual  actions  associated  with  an  individual’s  at¬ 
tention,  memory,  and/ or  execution  technique.  Rather  then 
being  executed  by  conscious  thought,  skill-based  actions 
(along  with  their  corresponding  errors)  tend  to  be  executed 
with  little  conscious  effort.  From  a  training  perspective, 
then,  it  is  important  to  discover  what  skill-based  actions 
are  being  executed  erroneously  so  that  the  individual 
can  be  made  aware  of  the  problem.  Thus,  a  skill-based 
training  environment  (ideally  one  that  simulates  actual 
job  conditions)  is  necessary  to  practice  new  skills  so  that 
they  can  be  executed  without  much  conscious  thought. 
For  example,  if  an  OE  were  the  result  of  a  controller’s 
perceptual  scan,  then  an  appropriate  training  environ¬ 
ment  would  be  one  that  consists  of  scanning  activities. 
Similarly,  if  a  loss  of  memory  were  implicated  in  an  OE, 
then  an  appropriate  training  environment  would  be  one 
that  consists  of  job-related  memory  activities.  In  both 
cases  the  training  activities  should  provide  increasing 
levels  of  difficulty  until  the  prescribed  skills  are  performed 
routinely  under  actual  job  conditions. 

In  contrast  to  skill-based  errors,  decision  errors 
are  the  result  of  intentional  behaviors  that  proceed  as 
planned,  yet  the  plan  itself  proves  to  be  inadequate  for 
the  situation  at  hand.  In  HFACS,  there  are  three  types 
of  decision  errors:  procedural  errors,  poor  choices,  and 
problem-solving  errors;  all  of  which  require  conscious 
effort  to  execute.  From  a  training  perspective,  then, 
the  challenge  is  to  provide  exercises  that  will  improve  a 
person’s  ability  to  make  better  decisions  within  the  time 
constraints  imposed  by  the  job  setting.  For  example,  as¬ 
sume  that  an  OE  was  the  result  of  an  ATCS  issuing  an 
altitude  change  to  resolve  a  potential  conflict,  but  in  so 
doing  created  a  loss  of  separation  with  another  aircraft. 
After  investigating  the  OE,  it  may  be  determined  that  the 
ATCS  would  benefit  from  real-time  training  activities  that 
required  him/her  to  predict  the  consequences  of  various 
actions  (i.e.,  speed,  heading,  and  altitude  changes)  used 
to  prevent  a  potential  loss  of  separation. 

Although  fewer  causal  factors  were  classified  as  decision 
errors  than  skill  based  errors,  considerable  attention  has 
been  placed  on  using  technology  to  aid  the  controller’s 
decision  process.  For  example,  decision  support  tools 
now  exist  that  provide  controllers  with  the  capability 
of  modeling  the  best  route  through  a  sector  or  within  a 
sector  so  as  to  minimize  interference  with  other  aircraft 
routes  in  adjacent  sectors  and  maximize  the  ability  to 
detect  potential  conflicts.  One  might  expect  that  with 
improved  decision  support  tools  there  would  be  fewer 
OEs.  It  should  be  noted,  however,  that  by  design  decision 
tools  are  directed  at  aiding  controller  decisions.  Thus, 
the  expected  reduction  in  OEs  should  come  from  those 
errors  that  are  the  result  of  poor  decisions.  However,  if 
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the  majority  of  OEs  result  from  skill-based  errors  (at  least 
based  on  the  results  found  in  this  analysis),  it  would  seem 
that  greater  attention  should  be  placed  in  this  area. 

One  area  of  training  that  has  recently  been  developed 
to  improve  ATC  skills  is  entitled  The  National  Air  Traffic 
Professionalism  (NATPRO)  project  (Pounds,  in  review). 
NATPRO  training  is  expected  to  improve  air  traffic  safety 
and  efficiency  by  increasing  the  controller’s  attention 
and  perception  skills.  The  training  consists  of  a  knowl¬ 
edge-based  seminar  followed  by  a  practicum  designed  to 
enhance  a  specific  skill,  for  example,  detection  of  relevant 
information  while  scanning.  Although  the  skill  itself  is 
generic,  the  specific)  ob  application  is  associated  with  radar 
scanning  in  the  en  route  and  terminal  radar  environments 
and  scanning  for  aircraft,  vehicles  and  pedestrians  from 
the  tower.  The  strength  of  NATPRO  training  is  that  par¬ 
ticipants  receive  immediate  feedback  on  their  performance 
relative  to  their  individual  baseline  performance  and  can 
request  feedback  about  their  performance  relative  to  the 
performance  of  other  participants. 

Returning  to  the  system  level  of  analysis,  one  of  the 
major  contributions  of  HFACS  is  the  labeling  (thus 
removing  the  mystery)  of  the  more  common  types  of 
latent  errors  (Reason,  1990).  However,  it  is  one  thing 
to  know  that  a  particular  latent  error  exists  within  the 
system,  but  that  does  not  tell  us  the  probability  that  a 
given  latent  error  might  occur.  It  would  be  useful  to 
develop  probability  estimates  for  the  occurrence  of  each 
type  of  latent  error  so  that  an  overall  OE  risk  assessment 
could  be  assigned,  e.g.,  to  a  given  ATC  facility,  a  specific 
sector,  or  particular  runway.  Considering  that  ASHRAM 
was  derived  from  human  reliability  risk  assessment,  the 
thought  of  integrating  HFACS  into  an  ASHRAM-like 
framework  has  great  appeal. 

The  existing  OE  investigation  process  could  be  further 
refined  to  better  identify  human  error  mechanisms  related 
to  OEs.  A  revised  Form  7210-3  could  then  be  used  to 
record  unsafe  or  contributing  actions  associated  with 
OEs.  Finally,  for  each  ATC  position  (e.g.,  radar  control 
or  local  control)  the  context  could  be  specified.  Having 
all  three  components  in  place,  the  final  step  would  be  to 
derive  an  overall  risk  assessment  described  previously  for 
a  given  facility,  sector,  runway,  and  so  on.  Although  the 
mathematics  involved  in  computing  a  risk  assessment 
are  beyond  the  scope  of  this  paper,  the  HRA  literature 
provides  the  necessary  guidance  for  completing  such  a 
project  (c.  f  Swain  &  Guttmann,  1983). 


CONCLUSION 

HFACS  proved  to  be  a  useful  taxonomy  for  classifying 
the  causal  factors  associated  with  OEs.  A  greater  percent¬ 
age  were  classified  as  skill-based  errors  as  compared  to 
decision  errors.  In  addition,  our  results  demonstrated  that 
the  "causal  factors"  listed  in  the  current  OE  reporting 
system  is  lackingin  information  concerning  organizational 
factors,  unsafe  supervisory  acts,  and  the  preconditions 
of  unsafe  acts.  It  is  recommended  that  greater  attention 
be  placed  on  developing  a  more  comprehensive  human 
factors  assessment  of  OE  causes  across  all  levels. 

As  with  any  study,  when  interpreting  these  results, 
one  should  consider  the  quantity  and  quality  of  the 
OE  data  available.  Any  post  hoc  analysis  depends  on 
the  comprehensiveness  and  accuracy  of  the  data.  In  this 
study  we  did  not  distinguish  between  facility  types,  i.e.. 
Air  Traffic  Control  Towers  (ATCTs),  Terminal  Radar 
Approach  Control  facilities  (TRACONs),  or  Air  Route 
Traffic  Control  Centers  (ARTCCs).  Also,  we  did  not 
examine  the  causal  factors  based  on  who  was  deemed  to 
be  primarily  responsible  for  the  OE  versus  who  played  a 
contributing  role.  We  also  used  only  a  subset  of  the  data 
from  the  7210-3. 

Since  the  nature  of  this  study  was  to  identify  a  candi¬ 
date  taxonomy  or  model  and  then  to  test  the  strongest 
candidate  using  OE  data,  the  above  limitations  do  not 
weaken  the  conclusion  that  HFACS,  or  some  variation 
of  it,  could  profitably  be  incorporated  into  the  OE  re¬ 
porting  process.  However,  for  those  who  wish  to  draw 
additional  conclusions  from  the  material  presented,  the 
above  limitations  should  be  considered. 
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APPENDIX  A 

HFACS  Categories  and  Definitions  (Wiegmann  &  Shappell,  2003). 

Unsafe  Acts  -  Performed  by  the  operator 

Errors  -  The  mental  or  physieal  activities  of  individuals  that  fail  to  achieve  their 
intended  outcome 

Skill  Based  -  Basic  skills  that  occur  without  significant  conscious 
thought 

Decision  -  Intentional  behavior  that  proceeds  as  planned,  yet  the  plan 
itself  proves  inadequate  or  inappropriate  for  the  situation.  This  is 
sometimes  referred  to  as  “honest  mistakes” 

Perceptual  -  Occurs  when  one’s  perception  of  the  situation  differs  from 
reality  usually  because  of  degradation  of  sensory  input 

Violations  -  Willful  disregard  for  the  rules  and  regulations 

Routine  -  Habitual  by  nature  and  often  tolerated  by  governing  authority 

Exceptional  -  Isolated  departures  from  authority  not  necessarily 
indicative  of  an  individual’s  behavior  pattern,  nor  condoned  by 
management 

Preconditions  for  Unsafe  Acts  -  Addresses  the  underlying  causes  for  Unsafe  Acts 

Environmental  Factors  -  Surrounding  conditions  affecting  the  operator 

Physical  Environment  -  Refers  the  operational  environment  and  the 
ambient  environment 

Technological  Environment  -  Encompasses  a  variety  of  issues  including 
the  design  of  the  equipment  and  controls,  display/interface 
characteristics,  checklist  layouts,  task  factors  and  automation 

Condition  of  Operators  -  Factors  within  the  individual  that  interferes  with 
optimal  performance 

Adverse  Mental  States  -  Preexisting  mental  states  (such  as  mental 
fatigue,  personality  traits,  pernicious  attitudes,  and  misplaced  motivation) 
affecting  performance 

Adverse  Physiological  States  -  Those  medical  or  physiological  conditions 
that  preclude  safe  operations 
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Physical/Mental  Limitations  -  Those  instances  when  operational 
requirements  exceed  the  capability  of  the  individual 

Personnel  Factors  -  Factors  that  individuals  do  to  themselves  to  create 
preconditions  for  unsafe  acts 

Crew  Resource  Management  -  Addresses  factors  that  lead  to  poor 
coordination  among  personnel 

Personal  Readiness  -  Addresses  situations  when  individuals  fail  to 
prepare  physically  or  mentally  for  duty 

Unsafe  Supervision  -  Supervisory  actions  that  influence  the  conditions  of  the  operator 
and  the  type  of  environment  in  which  they  operate 

Inadequate  Supervision  -  Deals  with  lack  of  guidance  and  oversight  in  day-to- 
day  operations 

Planned  Inappropriate  Operations  -  Occurs  when  individuals  are  put  in  an 
unacceptable  risk  due  to  the  operation  tempo  and/or  scheduling  of  work 

Failure  to  Correct  Problem  -  Those  instances  when  efficiencies  among 
individuals,  equipment,  training,  or  other  related  safety  errors  are  “known”  to  the 
supervisor,  yet  are  allowed  to  continue  unabated 

Supervisory  Violations  -  Those  instances  when  existing  rules  and  regulations  are 
disregarded  by  supervisors 

Organizational  Influences  -  Fallible  decisions  of  upper-level  management  affecting 
supervisory  practices  as  well  as  the  conditions  and  actions  of  the  operator 

Resource  Management  -  Encompasses  the  realm  of  corporate-level  decision¬ 
making  regarding  the  allocation  and  maintenance  of  organizational  assets  (such 
as  personnel,  money,  equipment,  and  facilities) 

Organizational  Climate  -  The  working  atmosphere  within  the  organization 
which  includes  culture,  policies,  and  structure 

Organizational  Process  -  Refers  to  corporate  decisions  and  rules  that  govern  the 
everyday  activities  within  the  organization.  This  includes  the  establishment/  use 
of  standard  operational  procedures,  and  formal  methods  for  maintaining 
oversight  of  the  workforce 


A2 


APPENDIX  B 


Probability  Risk  Assessment,  Human  Reliability  Analysis  and  ATHEANA 

A  Technique  for  Human  Error  Analysis  (ATHEANA)  belongs  to  the  class  of  human  reliability 
analysis  (HRA)  models  used  in  probabilistic  risk  assessments  (PRAs)  to  assess  the  “implications  of 
various  aspects  of  human  performance  on  risk”  (Cooper,  et.  ah,  1996,  p.  1-1).  Closely  tied  to  the  nuclear 
industry,  the  concept  of  PRA  was  developed  as  a  means  to:  (1)  identify  potential  areas  of  risk  and  indicate 
how  those  risks  could  be  mitigated,  and  (2)  quantify  the  overall  risk  potential  of  a  given  plant  (Reason, 
1990).  Both  criteria  were  necessary  to  achieve  public  and  government  support  for  the  building  and 
operating  of  a  nuclear  power  plants. 

As  a  guideline,  the  basic  steps  for  conducting  a  PRA  were  formally  proposed  in  a  1975 
publication  of  the  U.S.  Reactor  Safety  Study  known  as  WASH-1400  (Yellin,  1976).  Reason  (1990) 
summarized  the  PRA  steps  reported  in  WASH- 1400  as: 

a.  Identify  the  sources  of  potential  hazard. 

b.  Identify  the  initiating  events  that  could  lead  to  this  hazard. 

c.  Establish  the  possible  sequences  that  could  follow  from  various  initiating  events  using  event 
trees. 

d.  Quantify  each  event  sequence.  This  involves  data  or  judgment  about  two  things:  (1)  the 
frequency  of  the  initiating  event,  and  (2)  the  probability  of  failure  on  demand  of  the  relevant 
safety  systems. 

e.  Determine  the  overall  plant  risk.  This  will  be  a  function  of  the  frequency  of  all  possible 
accident  sequences  and  their  consequences  (p.  2 1 9) 

Although  PRAs  provided  essential  information  for  assigning  risk  to  a  proposed  nuclear  power 
plant,  they  failed  to  adequately  address  the  human  element  association  with  system  failures  (Reason, 
1990).  Specifically,  the  means  by  which  probabilities  were  assigned  to  potential  human  failures  lacked 
methodological  rigor.  Hence  a  need  arose  to  incorporate  a  Human  Reliability  Analysis  (HRA)  into  PRAs. 

HRA  is  defined  as  a  set  of  methods  that  assess  the  probability  that  a  person  will  correctly  perform 
some  system-required  activity  during  a  given  time  period  without  performing  any  extraneous  activity  that 
can  degrade  the  system  (Hollnagel,  2002).  One  of  the  more  widely  used  methods  of  conducting  a  HRA  is 
the  Technique  for  Human  Error  Rate  Predictions,  THERP  (Swain  &  Guttmann,  1983).  The  procedures 
used  by  THERP  are  similar  to  those  used  in  a  standard  PRA  and  include: 

a.  Identify  the  system  functions  that  may  be  influenced  by  human  error. 

b.  Conduct  a  detailed  task  analysis  of  the  related  human  operations. 

c.  Estimate  the  relevant  error  probabilities  using  a  combination  of  subject  matter  experts 
(SMEs)  and  empirical  data. 

d.  Estimate  the  effects  of  human  errors  on  system  failure  events  usually  by  integrating  HRA 
with  PRA. 

Although  widely  used.  Reason  (1990)  notes  that  THERP  is  often  criticized  for  its  simplistic  view 
of  the  human  operator.  THERP,  as  with  other  HRA  techniques,  assumes  that  the  human  operator  behaves 
as  any  other  piece  of  equipment.  That  is,  the  operator  either  performs  correctly  or  does  not.  This 
assumption  is  embedded  in  the  basic  formula  used  to  calculate  the  probability  of  a  specific  erroneous 
action  (Pea): 

N 

Pea  =  HEPea*  ^ 

k=\ 

where:  HEPea=  the  human  error  probability  of  a  erroneous  action 
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PSF  =  a  performance  shaping  conditions 
W  =  weight  of  a  PSF 
C  =  constant 
N  =  number  of  PSFs 

As  Flollnagel  (2000)  explains,  the  above  equation  contains  two  fundamental  assumptions,  neither 
of  which  is  reasonable: 

1)  First,  that  the  probability  of  failure  can  be  determined  for  specific  types  of  actions 
independently  of  any  context. 

2)  Second,  that  the  effects  of  the  context  are  additive,  which  is  the  same  as  saying  that  the 
various  performance  conditions  (such  as  interface  quality,  stress,  level  of  training,  complexity 
of  task,  etc.)  do  not  influence  one  another  (p.  2). 

Further  criticisms  of  FIRA  techniques  emerged  throughout  the  1980s.  The  major  criticisms  were 
summarized  by  Doughtery  (1990)  and  latter  elaborated  upon  by  Flollnagel  (2002)  as  follows: 

a.  Existing  empirical  data  are  insufficient  to  support  quantitative  predictions  of  human 
performance  in  complex  systems.  This  problem  had  actually  been  recognised  by  FIRA 
practitioners  since  the  early  1960s.  As  alternatives  to  empirical  data,  FIRA  often  relied  on 
either  expert  judgement  or  data  from  simulator  studies. 

b.  Expert  judgements  can  be  used  in  lieu  of  empirical  data,  but  there  is  a  lack  of  agreement 
about  the  use  of  expert  judgement  methods.  The  methods  neither  have  satisfactory  between- 
expert  consistency,  nor  produce  accurate  predictions. 

c.  Data  from  simulator  studies  can  be  used  instead  of  empirical  data,  but  the  calibration  to  real 
life  situations  or  ability  to  generalize  is  inadequate.  The  veracity  and  validity  of  simulator 
data  have  not  yet  been  convincingly  demonstrated. 

d.  The  accuracy  of  predictions  from  FIRA  methods  is  debatable  and  generally  unproven, 
particularly  for  non-routine  tasks.  Benchmark  studies  usually  produce  divergent  results,  for 
many  different  reasons. 

e.  The  psychological  realism  in  most  FIRA  methods  is  inadequate,  and  the  assumptions  about 
human  behaviour  are  often  highly  questionable  from  a  psychological  point  of  view. 

f  The  treatment  of  important  Performance  Shaping  Factors  is  inadequate.  In  particular,  there  is 
too  little  emphasis  on  PSFs  relating  to  management,  organisation,  culture,  etc.  (p.  2) 

Given  the  mounting  criticisms  surrounding  the  method  of  calculating  human  reliability 
probabilities,  researchers  began  developing  more  comprehensive  HRA  models,  such  as:  A  Technique  for 
Human  Error  Assessment  Early  in  Design,  THEA  (Pocock,  Wright,  and  Harrison,  1999);  Dynamic 
Reliability  Technique  for  Error  Assessment  in  Man-Machine  Systems,  DREAMS  (Cacciabue,  1993);  and 
the  Technique  for  Human  Error  Analysis,  ATHEANA  (Cooper  et  ah,  1996).  The  latter  will  be  discussed 
in  more  detail. 

The  goal  of  the  ATHEANA  project  was  to  develop  an  improved  method  for  HRA  that  would 
“allow  for  a  more  realistic  assessment  and  representation  of  the  human  contribution  to  plant  risk,  and 
thereby  increase  the  utility  of  PRA”  (Cooper  et  al,  1996,  p.  ix).  Specifically,  ATHEANA  was  developed 
to  address  deficiencies  in  HRA  approaches  by: 

1 .  Addressing  errors  of  commission  and  dependencies 

2.  Representing  more  realistically  the  human-system  interactions  that  have  played  important 
roles  in  accident  response,  as  evidenced  by  operating  experience,  and 

3.  Integrating  recent  advances  in  psychology  with  engineering,  human  factors  and  PRA 
disciplines  (Cooper  et  al.,  1996,  p.  1-2). 
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The  above  were  aeeomplished  through  an  integration  of  retrospective  analysis  of  past  operating 
events  and  the  prospective  analysis  in  support  of  PRA.  This  can  be  seen  in  the  six  elements  comprising 
ATHEANA  as  shown  below.  The  six  elements  fall  under  three  broad  categories  consisting  of: 

1.  Error  Forcing  Context  (EEC) 

a.  Performance  Shaping  Factors  (PSF) 

b.  Plant  Conditions  (PC) 

2.  Human  Error 

a.  Error  Mechanisms 

b.  Unsafe  Actions 

3.  PRA  Model 

a.  Human  Failure  Events  (HFEs) 

b.  Scenario  Definitions 

EEC  represents  the  interaction  of  PC  and  PSF.  Typically  EFCs  are  unanalyzed  plant  conditions 
(similar  to  Reason’s  latent  errors)  that  serve  to  shape  an  operator’s  response  to  an  event.  Examples  of 
unanalyzed  plant  conditions  are  the  effects  of  a  history  of  false  alarms  associated  with  a  system 
component,  shutting  off  safety  functions  during  various  phases  of  system  shut  down,  and  unusual  or 
incorrect  equipment  configuration  that  is  integrated  into  the  operating  system.  Examples  of  performance¬ 
shaping  factors  include  such  factors  as  procedures,  training,  communications,  supervision,  staffing, 
organizational  factors,  stress  and  environmental  conditions. 

ATHEANA  defines  human  error  as  the  “divergence  between  an  action  actually  performed  and  the 
action  that  should  have  been  performed”  (Cooper  et.  ah,  1996,  p.2-9).  Although  the  term  “action”  implies 
a  behavioral  act  (i.e.  unsafe  actions),  ATHEANA  also  allows  for  error  mechanisms  (i.e.,  psychological 
mechanisms  such  as  judgment  and  decision  making)  to  be  incorporated  into  the  definition  of  human  error. 

The  PRA  model  is  based  on  probability  estimates  for  the  frequency  of  occurrence  of  specific 
failure  points  associated  with  a  given  accident  scenario.  HFEs  represent  the  failure  of  a  function,  system 
or  component  as  the  result  of  an  unsafe  action  by  the  human  operator  (e.g.  errors  of  commission  and 
errors  of  omission)  that  places  the  operation  in  a  worse  condition.  The  scenarios  that  are  used  to  develop 
probability  estimates  usually  consist  of  event  tree  and  fault  tree  sequences  that  model  a  particular  chain  of 
events.  The  level  of  detail  presented  depends  of  the  function,  systematic,  or  component  level  of  analysis. 
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APPENDIX  C 

Form  7210-3 


Final  Operational  Error/Deviation  Report 

Report  Number 

Part  II.  FACILITY  MANAGER  ACTION 

67.  Select  the  category  of  the  operational  error/deviation,  (more  than  one  category  may  be  possible) 

□  Procedural  O  Equipment  □  ATCS  O  Manager/Supervisor/Other  Personnel 

68.  Causal  Factors 

No 

Yes  (employee) 

A 

B 

C 

D 

E 

A.  Data  Posting 

□ 

(1)  Computer  Entry 

□ 

Incorrect  input 

□ 

□ 

□ 

□ 

□ 

Incorrect  update 

□ 

□ 

□ 

□ 

□ 

Premature  termination  of  data 

□ 

□ 

□ 

□ 

□ 

Input/Update  not  made 

□ 

□ 

□ 

□ 

□ 

Other  (explain) 

□ 

□ 

□ 

□ 

□ 

(2)  Flight  Progress  Strip 

□ 

Not  updated 

□ 

□ 

□ 

□ 

□ 

Interpreted  incorrectly 

□ 

□ 

□ 

□ 

□ 

Posted  incorrectly 

□ 

□ 

□ 

□ 

□ 

Updated  incorrectly 

□ 

□ 

□ 

□ 

□ 

Premature  removal 

□ 

□ 

□ 

□ 

□ 

Other  (explain) 

□ 

□ 

□ 

□ 

□ 

B.  Radar  Display 

□ 

(1)  Misidentification 

□ 

Failure  to  reidentify  aircraft  when  the  accepted  target  identity  becomes  questionable 

□ 

□ 

□ 

□ 

□ 

Overlapping  data  blocks 

□ 

□ 

□ 

□ 

□ 

Acceptance  of  incomplete  or  difficult  to  correlate  position  information 

□ 

□ 

□ 

□ 

□ 

Other  (explain) 

□ 

□ 

□ 

□ 

□ 

(2)  Inappropriate  Use  of  Displayed  Data 

□ 

MODE  C 

□ 

□ 

□ 

□ 

□ 

BRITE 

□ 

□ 

□ 

□ 

□ 

Conflict  alert 

□ 

□ 

□ 

□ 

□ 

Failure  to  detect  displayed  data 

□ 

□ 

□ 

□ 

□ 

Failure  to  comprehend  displayed  data 

□ 

□ 

□ 

□ 

□ 

Failure  to  project  future  status  of  displayed  data 

□ 

□ 

□ 

□ 

□ 

Other  (explain) 

□ 

□ 

□ 

□ 

□ 

C.  Aircraft  Observation  (Towers  Only) 

□ 

(1)  Actual  Observation  of  Aircraft 

□ 

□ 

□ 

□ 

□ 

(2)  Improper  Use  of  Visual  Data 

□ 

Landing 

□ 

□ 

□ 

□ 

□ 

Taking  Off 

□ 

□ 

□ 

□ 

□ 

Ground  Operation 

□ 

Taxiing  across  runway 

□ 

□ 

□ 

□ 

□ 

Holding  in  position  for  takeoff 

□ 

□ 

□ 

□ 

□ 

Other  (explain) 

□ 

□ 

□ 

□ 

□ 

Cl 


No 

Yes  (employee) 

A 

B 

C 

D 

E 

D.  Communication  Error 

□ 

(1)  Phraseology 

□ 

□ 

□ 

□ 

□ 

(2)  Transposition 

□ 

□ 

□ 

□ 

□ 

(3)  Misunderstanding 

□ 

□ 

□ 

□ 

□ 

(4)  Read  back 

□ 

Altitude 

□ 

□ 

□ 

□ 

□ 

Clearance 

□ 

□ 

□ 

□ 

□ 

Identification 

□ 

□ 

□ 

□ 

□ 

Other  (explain) 

□ 

□ 

□ 

□ 

□ 

(5)  Acknowledgement 

□ 

□ 

□ 

□ 

□ 

(6)  Other  (explain) 

□ 

□ 

□ 

□ 

□ 

E.  Coordination 

□ 

(1)  Area  of  Incident 

□ 

Intra-sector/position 

□ 

□ 

□ 

□ 

□ 

Inter-sector/position 

□ 

□ 

□ 

□ 

□ 

Inter-facility 

□ 

□ 

□ 

□ 

□ 

Facility  type:  .level:  ,  and  facility  ID: 

□ 

□ 

□ 

□ 

□ 

(2)  Failure  to  utilize/comply  with  precoordination  information 

□ 

□ 

□ 

□ 

□ 

(3)  Improper  use  of  information  exchanged  in  coordination 

□ 

Aircraft  Identification 

□ 

□ 

□ 

□ 

□ 

Altitude/Flight  Level 

□ 

□ 

□ 

□ 

□ 

Route  of  Flight 

□ 

□ 

□ 

□ 

□ 

Speeds 

□ 

□ 

□ 

□ 

□ 

APREQs 

□ 

□ 

□ 

□ 

□ 

Special  Instructions 

□ 

□ 

□ 

□ 

□ 

Other  (explain) 

□ 

□ 

□ 

□ 

□ 

(4)  Failure  to  coordinate  between  ground  and  local  control 

□ 

Crossing  active  runway 

□ 

□ 

□ 

□ 

□ 

Vehicle,  equipment,  or  personnel  on  active  runway 

□ 

□ 

□ 

□ 

□ 

Use  of  other  than  active  runway  for  arrival  and  departures 

□ 

□ 

□ 

□ 

□ 

Runway  closure 

□ 

□ 

□ 

□ 

□ 

Other  (explain) 

□ 

□ 

□ 

□ 

□ 

F.  Position  Reiief  Briefing 

□ 

(1)  Employee  did  not  use  position  relief  checklist 

□ 

□ 

□ 

□ 

□ 

(2)  Employee  being  relieved  gave  incomplete  briefing 

□ 

□ 

□ 

□ 

□ 

(3)  Relieving  employee  did  not  make  use  of  pertinent  data  exchanged  at  briefing 

□ 

□ 

□ 

□ 

□ 

(4)  Other  (explain) 

□ 

□ 

□ 

□ 

□ 
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FORM  7210-3  DEFINITIONS 


SECTION  A:  DATA  POSTING 

A  data  posting  error  is  any  error  of  calculation,  omission,  or  incomplete  data,  erroneous  entries,  handling,  or  subsequent  revisions 
to  this  data.  This  includes  errors  in  posting  and  recording  data.  It  does  not  include  errors  involved  in  receiving,  transmitting, 
coordinating,  or  otherwise  forwarding  this  information.  If  one  of  the  causal  factors  listed  does  not  adequately  describe  the  factor 
involved,  list  the  factor  under  "Other"  and  provide  a  brief  explanation. 

SECTION  B:  RADAR  DISPLAY 

a.  Misidentification 

Radar  misidentification  means  a  failure  to  properly  identify  the  correct  target  and  includes  subsequent  errors  committed  after  the 
original  identification  was  properly  accomplished.  Indicate  the  listed  item(s),  which  most  closely  describes  the  reason  for 
misidentification.  If  one  of  the  causal  factors  listed  does  not  adequately  describe  the  factor  involved,  list  the  factor  under  "Other" 
and  provide  a  brief  explanation. 

b.  Inappropriate  Use  of  Displayed  Data 

A  data  or  display  information  error  occurs  due  to  a  failure  to  maintain  constant  surveillance  of  a  flight  data  display  or  traffic 
situation  and  to  properly  use  the  infomiation  presented  by  the  display  or  situation.  If  one  of  the  causal  factors  listed  does  not 
adequately  describe  the  factor  involved,  list  the  factor  under  "Other"  and  provide  a  brief  explanation. 

SECTION  C:  AIRCRAFT  OBSERVATION  (Towers  Only) 

An  aircraft  observation  error  means  a  failure  to  maintain  constant  surveillance  of  aircraft  and  the  movement  area,  and  to  properly 
react  to,  interpret,  or  otherwise  utilize,  in  a  timely  manner,  the  infonnation  being  viewed.  If  one  of  the  causal  factors  listed  does 
not  adequately  describe  the  factor  involved,  list  the  factor  under  "Other"  and  provide  a  brief  explanation. 

SECTION  D:  COMMUNICATIONS  ERROR 

A  communications  error  is  a  causal  factor  associated  with  the  exchange  of  infomation  between  two  or  more  people  (e.g.,  pilots 
and  specialists).  It  refers  to  the  failure  of  human  communication  not  communications  equipment. 

a.  Phraseology 

Use  of  incorrect  or  improper  phraseology. 

b.  Transposition 

An  error  due  to  transposition  of  words,  numbers,  or  symbols  by  either  oral  or  written  means.  This  involves  writing/saying  one 
thing  while  thinking/hearing  something  else. 

c.  Misunderstanding 

The  failure  to  communicate  clearly  and  concisely  so  that  no  misunderstanding  exists  for  any  actions  contemplated  or  agreed 
upon. 

d.  Read  back 

The  failure  to  identify  improper  or  incorrect  read  back  of  information. 

e.  Acknowledgment 

The  failure  to  obtain  or  give  an  acknowledgment  for  the  receipt  of  information. 

f.  Other 

If  the  causal  factors  listed  above  do  not  adequately  describe  the  factor  involved,  list  the  factor  and  provide  a  brief  explanation. 
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SECTION  E:  COORDINATION 


Any  factor  associated  with  a  failure  to  exchange  requirement  information.  This  includes  coordination  between  individuals, 
positions  of  operation,  and  facilities  for  exchange  of  information  such  as  APREQ's,  position  reports,  forwarding  of  flight  data, 
etc.  If  one  of  the  causal  factors  listed  does  not  adequately  describe  the  factor  involved,  list  the  factor  under  "Other"  and  provide  a 
brief  explanation. 


SECTION  F:  POSITION  RELIEF  BRIEFING 

Relief  briefing  errors  are  special  errors  of  both  communication  and  coordination,  which  occur  as  the  result  of  position  relief 
They  include  such  things  as  failure  to  give  a  relief  briefing,  failure  to  request  a  briefing,  incomplete  or  erroneous  briefing,  etc.  If 
one  of  the  causal  factors  listed  does  not  adequately  describe  the  factor  involved,  list  the  factor  under  "Other"  and  provide  a  brief 
explanation. 
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